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HUMNAHY 


An i>X|HT Itiu'nwal uvaluntlun ol tlic iiiun I luti'd .'ipproarli priu'cilm i' im i nn 
duotliiR low vlH'lblHty Inuti-umi'iil: npproaclicn la iloiict I lied . Foiif ali'lur i ii vai 
oadi flow 1(» appi’oadioH imlnp, tiio iiiotvl torod pvoi'cdiiro and l(i ualnp, a mud 1 1 ltd 
"Htandard" proroduro In a IKI-IO Hlmulator undiT varloun comHt l(iii!i lO "I, '.Hill 
tty, wind Hlionr and turbiFlonro, and radar vector Inc. Hcon.'ir Ion , 

tn tonmi of Hyntoni moamiroH of aircrew pi'rformanco, no malor d I n erenci'!! 
wore found. Pilot opinion data Indicate that there are Home denhahle eh.ir.ie 
torlHttcH of the monitored procedure, particularly with relerence lo tin' 
incrcnaod ro.lu of the flight engineer in conductlnc, low vlKlblllty a|ipro.icln ";. 
Rationale for developing approach procodurea la dlacuaaed. 



TNTUOmiCTlON 


The research described In thl.a paper grew out of some of the ctniccrn;. 
expressed by airline pilots during the preliminary pilot Interview stud lea 
mentioned In the previous paper (ref. 1). Specifically, many pilots felt 
that the approach procedures they were trsing were less than optimal willi 
regard to two major items: (a) the Integration of all three eockiilt crew- 

members into the approach procedure; and (b) the callouts roqulred of i he 
various crewmembers during an approach. 

The critical demands placed upon pilots during the last one or two hundred 
feet of an approach are well known. Many accidents have occurred during tills 
critical phase of flight, and in many of these It appears that one of thi^ m.iior 
contributing factors was the inadequate or inappropriate design .'f ap|no;ieh 
procedures, Including crew Integration and callouts. Tn many ca:?es siirflcient 
information to prevent the impending disaster was present within the cocki'll, 
yet the crew failed to utilise this information. Once the flying pilot has 
changed to flight by visual reference, deviations from the desired rUght p.ii li 
might not be readily discernible from outside visual cues. Tbese dcvl.il iuir. 
will, however, invariably show up on the coekpit Instnimeui at Inn : In. i e.i;;ed 
sink rate, deviations below glide slope, or low airspeed all so-caM.'.l 
killer ltem.s — will be displayed Inside tlie cockpit. Tt Is iiecess.irN lli.it this 
information be transferred to the flying, pilot If an ;ieeldi-nl is to hu 
prevented. 

There are two ways of performing thl.s t/iskt (a) the physiiil .nv i ■ in'iii. ni 
can bo modified, making the information available In the pilot's vicuii li. M 

'.'i| 



I 

[ 


I 


vin tiu> UMO nl VMil or n Uond-llp tllH|).lay, fur (>xamp1o; or, (b) tbo opuratr-onaX 
t'liv 1 ronnic’iit ran I'c modi flod Ijy uHlnp, aufolaiid or by tl)o adoption of now call"’ 
(jutH and monitor I np, proooduroH. 

Plguro 1 Il hiiitratoH tin* rolat lormb Ip bulwcon tln> proaont exporJinont and 
tno fi)ldc’mI<jlu}>loal raodol donerlbod In tlu* provlouH paper by lUlllngn et aX. 
(rof. 1). Tlu' primary lntor(>Ht wan in tln> t'ffoi't.n of manipulation of tho 
operational e.nvl ronnunit; , np(>e I f I ea I ly low vlnlbllity approae.b procedures, upon 
aircrew and aircraft performance. 

Although appr(>acli procedur<(H used by airlines vary widely, It Is possible 
to discern two basU' i^hilosopbles which have been used to structure these pro- 
cedures. One of these, the standard procedure, is basically this: one pilot 

is responsible for flying the approach and landing, or missed approach if that 
should be necessary, and the other crewmembers are assigned monitoring and 
callout duties. The decision to land or to go around is made by the flying 
pilot on the basis of his assessment of the visual situation following the 
transition from heads-down flying. Variations of this basic procedure are 
used by virtually all U.S. air carriers. 

One alternative to the standard procedure is one called the monitored 
approach by several of the foreign carriers who have developed these tech- 
niques, Basically, using this procedure, one pilot, usually the copilot, is 
i-esponslble for flying the heads-down portion of the approach; the other 
pilot is responsible for monitoring this portion o/ the approach and is the 
individual who decides whether the outside visual cues are sufficient for the 
landing. If they are, this pilot, who is usually the captain, takes physical 
control of the aircraft and proceeds with the landing. At the transition, 
the copilot assumes responsibility for monitoring the remainder of the approach 
and landing, remaining head down until si>metime during the landing roll. 

Intuitively, this monitored approach procedure has some appealing fea- 
tures, particularly In the way the transition from Instrument reference to 
visual reference flight Is made. The eaptain Is given sufficient time to 
assess the vlsu;il situation and reach a decision and can do so without the 
additional burdcui of flying the aircraft. Furthermore, more emphasis is placed 
upon continuous monitoring of the critical final portion of the approach and 
landing. However, there are also some characteristics of this procedure which 
appear to be less desirable, particularly those having to do with the physical 
transfer of aircraft control at very low altitudes. 

In attempting to resolve these and other issues, It soon became apparent 
that there Is little. If any, objective data pertaining to the relative effec- 
tiveness of these two basic philosophies for conducting low visibility 
approaches. On the basts of the accumulated operational experience of those 
carriers who have used the monitored procedure, it can he concluded that the 
Idea has considerable, merit. However, because of the fundamental importance 
of approach ))rnee (hires for the safety of aircraft operations, decisions to 
utilize this approach, or any other for that matter, should he based on more 




rigurouH, ubjoctivc' pprfonmmcf daln nbi/ilnutl from Iliu* pi hits npt*ral. 1 nj'. muha* 
a wide variety of rrvi'l lHt Ir (mmuI 1 1 lonn. 

In aianmaryi the objective of the ex)ier I ment wan to coKipan* aheraft 

and aircrew perh>rinnnce during low vintblUty a)»))n>a<‘l)eH uninp, Mltlier tl)r 
standard or the monitored approach procedure* A necondary ol)JectIvi* wais to 
develop full miHoion Hhimlatlon reHearch technlq\ieH for une hi otlu*r reneandi* 
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Dovolopnient of Approach broceduren 

Because the carrier wIk^ part 1 cipatt*d In the Htudy used a variant of the 
standard procedure, It was necessary to develop a modified standard procedure 
in order to control for the possible effects of crew familiarity wltli tlu‘ 
standard procedure* To accomplisli this, the approach procedures and ealhuits 
used by another U*S* carrier were used* This set of pn^cedures was suffi*- 
ciently different from those used by the participating carrier, that the 
likelihood that familiarity influenced the results of this study Is minimal* 
This set of procedures is summari^:ed in figure 2* 

Two major criteria were used during the development of the monitored 
procedure which was used in this experiment* (a) the flight engineer should 
be fully and completely integrated into the approach procedure; and, 

(b) there should always be a clear*-cut division of responsibilities — 
pilot flying, primary monitor, and backup monitor — as shown in figure 3* 

In other words, at any given point during an approach, each crewmember 
should be assigned one of these three functions, and whenever tliere is a 
change in one crewmember’s function, there should be a corresponding, com-* 
pensatory change in another crevrmiember ’ s function* Thus, for example, when 
the flight engineer calls out, "Approaching mlnlmums," the captain verbal Iv 
acknowledges this callout and changes to outside visual reference. Simul- 
taneously, the flight engineer assumes the primary monitoring duttes inside 
the cockpit, and the first officer continues to function as the flying pilot. 
Wlien the captain announces, "Land," the first officer now assumes primary 
monitor duties, the flight engineer resumes his role of hack-up monitor* and 
of course, the captain becomes the pilot flying. 

Callouts were constructed with regard to the three major !' unctions x>?]iich 
callouts can perfonn: (a) they serve to transmit information about tlie state 

of the aircraft; (b) they serve to check for subtle pilot incapae i 1 at i (Ui if 
a pilot misses a callout, or falls to acknowledge one, the oilier pi Inis sIumiIi! 
check to make sure the cpiiet one Is still witli tliem; and finallv, (c) cal Units 
can be used to help enforce heads-down discipline* If we want to maximi/.i’ tlie 
probability that a pilot will nniiain on the Instruments during the lust stap.es 
of an approach, we c;in assign him speeiiit' callout duties diirinp, tlial pi’rin<i 
of time* 
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MiHiU’cl nppronelic'H wero mitoiiiMtlc; If tlie cfiptnin had not taken control of 
the ;i I reran, wlii'ii It reached the mJHHcd approach point, the first officer 
hill laled tlua mlsHed approach procedure, and the captain came back inside, tlie 
ioc|<pll lo ref-mme the role of primary monitor. If It became necessary to p,o 
around alter the captain had decided to land, the captain called out, "Missed 
ap|>roach," and the first officer resumed control of the aircraft and announced, 
"I have the airplane." This procedure was chosen because it was reasoned that 
tlu> first officer, belnp, continuously heads down, was In the best position to 
asiiuine rapid and precise control of the aircraft. 

l''Jj>ore 4 shows the work sheet which was provided to the flight engineer 
when using the monitored procedure, before each approach was begun, the 
flight I’UgIneer was given an approach plate by the pilot so that he could 
detcumine the Information shown on the worksheet. This information was used 
subsequently by the flight engineer for cross checking and for callouts. In 
addition, the flight engineer was assigned very specific mo.iitoring duties and 
guldellni'S for calling out deviations from the desired flight profile. 


Subjects 

Because the study involved training airline pilot subjects on the use of 
an approach procedure which was not the approved procedure used by their 
company, training pilots, rather than line pilots, were used for this experi- 
ment. It was felt unwise to risk the possibility of training someone to the 
point where, if he were by chance to fly an actual low visibility approach 
shortly after his participation in this study, he might revert to the experi- 
mental procedures rather than use the approved procedure. 

Right current instructor pilots and four current flight engineer instruc- 
tors served as subjects for this experiment. These instructors were assigned 
to one of four crews. The flight experience of each of the subject pilots is 
summarized in table I. 


Simulation Facilities 

The simulator used for this experiment was a DC-10 simulator equipped 
with a six-degree-of-freedom motion platform and a TV, model-terrain- 
board visual system. Modifications were made to the simulation software to 
allow control of the experimental conditions from the Instructor’s CRT display 
and control panel located in the cockpit, and to allow real-time recording of 
simuJator data on digital magnetic tape. Additionally, provisions were made 
for recording communications, cockpit voice, and observer comments. Experi- 
mental sessions, each four hours long, were Intef ted Into the normal simu- 
lator training schedule. 


Simulator Scenarios and Experimental Design 

Since one of the primary areas of concern with the monitored procedure 
ccMitors around the question of transfer of control of the aircraft at low 
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aitUudi'H, only liaiul-l U'wn approaeliuK were- uhchI. Autopi lot uko nnd Uh 
liituract Ion wi th a|iproadi proc'diiroH Is a separate qucHtlon which was not 
.uidrcHscd In thin study. 


I’oHsIh li' Interact tons bctwc'cn the kind of approach and the approach pro- 
ccduri'H wore also oT Interest. Therelore both non-dlrectlonai beacon (NDB) 
and Instrument Jandinp, system (II.S) approaches were flown. Ail *S approaches 
were flown uslnp the Kl l}'.ht Oirector and manual throttles. AH Ni. approaches 
weri' flown usinp, raw data only. 


Each crew flew a total of 32 approaches during the data collection phase 
of the expe.r i ntont , sixteen usin}' tlu? monitored approach procedure, and sixteen 
using the standard procedure. 

Since one of the characteristics of a good set of approach procedures Is 
to better enable crews to cope with difficult operational situations, the 
effects of a variable called "Stress and Workload" on crew performance were 
evaluated. To accomplish this, radar vectoring techniques, wind shear, and 

were used to generate high and low stress and workload conditions. 

The low workload condition involved no turbulence, no wind shear, a five-knot 
crosswind from either the right or left, and radar vectoring service that was 
nearly ideal - timely, accurate, and such that the aircraft would intercept 
the final approach course well outside the final approach fix at the proper 
altitude and airspeed. In contrast, the high workload condition involved a 
forty-knot head or tail wind which sheared to a direct crosswind of ten knots 
by 61 m (200 ft) above ground level (ACL), some turbulence, and radar vector- 
ing of the kind too often encountered in the real world — late descent clear- 
ances, late turn-ons, and delayed speed reductions. These vectoring scenarios 
were chosen such that, if flown precisely, the aircraft would intercept the 
glide slope and final approach course right at the final approach fix (FAF) 
for the ILS approaches, and 1.6 km (1 mi) outside the final approach fix for the 
NDB approaches. These were difficult scenarios to fly, and they were chosen 
deliberately because instructor pilots are extremely proficient simulator 
pilots and it was necessary to ensure that there was ample opportunity for 
deviations from profile to develop. 

Each approach (as shown in fig. 5) was begun from identical conditions: 
downwind heading, l.V*2 ra (5000 ft) AGL, 250 knots, and with the aircraft in a 
clean configuration from a position 16 km (10 mi) abeam (either right or left) 
of the final approaco fix. After a prolimlnary briefing during which the 
approach location and type were specified, the simulator was released, and 
the Experiment Controller, a qualified DC— 10 instructor pilot who worked with 
us for the dui'at Ion of the study, proceeded to give radar vectors according 
to the presclectc'd scenario. .Standard company operating procedures. Including 
clu'cklists, were. usi*d lor all approaches. 


An appri)ach was terminated during the landing rollout, or upon ’-caching 
150 m (500 ft) ACl. dtiring the missed appri.iach. For half of the a’proaches, 
the simulated visibility was set to zero (below mlninmms). Fo’ tlio remaining 
half, the visibility was sel at the appropriate minlmums to’- the approach 
type, Daylight conditions were simulated in all cl rcumst._irices. 
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Two crewa flew the monitored approach procedurea flrat, fol lowed hy the 
atandard procedure. The order waa reveraed for the remalnlnf' two erewa. 

Data collection waa proceeded by a 2-hr Rround school session durlni', 
which two crews were briefed regarding the approach procedures they were about 
to fly. Following ground school, the pilots wore given a 1-hr. 15-mln simu- 
lator training session during which 4 ILS and 4 NDB approaehea were flown 
utilizing the appropriate set of procedures. The entire sequence of ground 
school, simulator training, and data collection was repeated for th<^ alt(!rnate 
set of approach procedures. Upon completion of the last data collection run, 
an extensive debriefing session was held during which comments, observations, 
and suggestions of the pilots were soi\'ht. 


RESULTS 


For the purposes of analyzing the tracking data recorded during this 
study, each approach is arbitrarily divided into two segments. The initial 
approach segment is that portion of the approach between the Final Approach 
Fix and a point 10 sec prior to reaching the missed approach point. The 
remainder of an approach to a landing Is termed the final approach segment. 
Landings and missed approaches were analyzed separately from the Initial 
approach data. This division was necessary to enable the analysis to focus 
clearly upon the critical last 100 m of an approach. For all practical pur- 
poses, there is little difference between the two kinds of approach procedures 

prior to the missed approach point. It is at the point where the control of 

the aircraft is transferred from one pilot to the other that major differences 

would be most likely to appear. ILS and NDB approaches were analyzed sepa- 
rately. 


Initial Approach Segment 

Tracking data were transformed into rms lateral error, rms glide-slope 
error, and airspeed variability measures, and were subjected to an Analysis 
of Variance. As expected, the stress and workload variable did significantly 
affect airspeed, localizer and glide-slope tracking for the ILS approaches, 
and lateral course error and airspeed control for the NDB approaciu*s. No 
other factor, including the set of approach procedures used, produced any 
significant differences in aircraft performance. 


Final Segment 

One measure, lateral error during NDB approaches, was slgnlflcanMy dif- 
ferent as a function of approa h procedure - lateral tracking was more varlahU 
using the monitored procedure. This was one of only two instances v<?here the 
approach procedure variable resulted In a significant difference in perf.umanc'. 
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LantUnpK wi^ra analyzed ualnp lateral and longitudinal error and alnK rate 
at touchdovm aa ratmourea of landlnp, performance , There were no alpnlf leant 
dlfferoncofl obaerved for any landing meaaureH. 


Miased Approach Data 

Mlaaod approach performiince was evaluated using peak deviation below 
MDA/DU (where MDA la minimum descent altitude and DH is decision height), and 
the square of peak deviation to give emphasis to the larger and presumably more 
dangerous deviations. In addition, the time integral of total flight path below 
MDA/DH was analyzed. The average peak deviation below MDA for NDB approaches 
was significantly larger using the monitored approach procedure. No other 
significant differences were observed. 

Debriefing Interview Results 

Pilot reaction during the training sessions to the monitored procedure 
was largely negative, and virtually all subjects expressed concern about the 
transfer of control of the aircraft. These negative attitudes were modified 
after the subjects had experience with the experimental set of procedures; 
however, it is still necessary to characterize the prevailing attitude as 
"concerned." Most pilots, however, did concede that there were some positive 
benefits to using the experimental procedure, particularly in reference to 
the Increased monitoring discipline achieved with this procedure. 

There was universal acclaim from the subjects for the increased emphasis 
on involving the flight engineer in the approach. It was the concensus that 
this one aspect of the experimental procedure was by far the most Important 
and valuable. 


DISCUSSION AND CONCLUSIONS 


In some ways the lack of major significant difference between the two 
procedures was a disappointing outcome. However, in retrospect, there are 
some encouraging aspects as well. 

First, with respect to the question of the superiority of one set of pro- 
cedures over another, it is necessary to conclude on the basis of results 
obtained here, that crews can perform equally well using either set of proce- 
dures. There Is no clear-cut reason to select one set of procedures over 
another on the basis of system performance measures used In tliis experiment. 
Put another way, the choice of which of the basic approach procedures to be 
used should be based upon other factors. Particularly important here is the 
accumulated experience of a company wltli one set of procedures; the difficul- 
ties encountered In changing from one set of procedures to anotlier may far 
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outwolf»h the potential ndvnntaRo.s obtained by adoptlnp, an alternate aet of 
procedures , 

Another conclusion la that reRardless of which basic approach procedure 
Is used, It Is Important that the fllRlit enRlneer be fully IntCRrated Into the 
approach. The callouts and monitor Inp, duties which were asslp.ned the fllp.ht 
onRlneer are larpely independent of the approach procedure adopted. AltliouRh 
not directly supported by the tracking data obtained In this study, there Is 
little doubt that this Is the most Important single consideration In tlie 
development of low visibility approach procedures. 

And finally, we can conclude that simulator evaluations of approach proce- 
dures are feasible. 

In summary, this first experiment was a preliminary attempt to assess the 
effects of selected operational factors on pilot performance, in this case 
with largely negative results. In a second study, the experience accumulated 
during this first study was used to refine procedures and techniques in an 
attempt to understand how certain perturbations in the operational environment 
can affect aircrew behavior. The preliminary results are highly encouraging, 
and it is intended to pursue the leads generated by those data in an attempt 
to see if techniques can be developed which will help airline pilots to cope 
with such disturbances. 
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TABLE I,- PILOT DATA 


CREW 

SUBJECT 

TOTAL TIME 

DC-10 


1 

9090 hr 

600 hrs 

A 

2 

14000 

400 


3 

1500 

400 


4 

14000 

400 

B 

5 

13000 

400 


6 

6000 

700 


7 

9000 

600 

C 

8 

11000 

400 


9 

15000 

300 


10 

7600 

500 

D 

11 

6000 

500 


12 

6900 

(2.5 yr ) 












